Performance of a Multi-class Biomedical Tagger on Clinical Records

We tested the performance of Cocoa, an existing dictionary/rule based entity tagger that tags multiple semantic types in biomedical domain including diseases, on disease/sign/symptom detection in clinical records in the ShARe/CLEF eHealth task. Initial analysis showed that the precision was high (≥ 90%), but recall was low (≈ 50%) due to (a) phrases peculiar to clinical notes (b) disambiguation of common words and (c) the large number of undefined acronyms. We extended the system to handle these cases by reference to the local intrasentential context as derived from the training set. A small module was also added for event-based detection of annotated sentence fragments containing verbs/gerunds; an example is ‘LV systolic function appears depressed’. The event detection system had about 30 rules. With these modifications, the f-score was 0.75 on the test set. In a second run, we added about 70 frequently occurring acronyms as well 15 phrases which were all in caps. The final results on the test set (f = 0.78) show that a multi-class tagger can work reasonably well on clinical records.